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ABSTRACT 

Field instruction is a critical piece of undergraduate geoscience majors' education, and fieldwork 
remains a major part of the work of professional geologists. Despite the central importance of field 
education, there exists relatively little educational research exploring how students learn to solve 
problems in geological fieldwork. This study adds tools and insight to the study of field problem 
solving. We used GPS tracking of students engaged in independent field examinations, and developed 
two parallel coding approaches for analyzing student navigational choices. Taken together, our coding 
enables correlation of navigational characteristics with performance and lends insight into problem 
solving by building on a conceptual framework modified from the cognitive science field of Naturalistic 
Decision Making. Our results indicate that most advanced geology undergraduates are capable of 
recognizing important features in the field, however lower-performing students fail to systematically 
test multiple interpretations of their data as reflected in poorly planned traverses across the examination 
field area. Specific track sequences, especially those involving reoccupation of locations, show particular 
difficulties in aspects of problem solving that are reflected in low quality interpretations on finished 
maps. Our study offers new tools and an independent approach to gauging student skills in geologic 
field problem solving. 


INTRODUCTION 

Field-based instruction is widely 
acknowledged to be a central part of 
undergraduate education in the geological 
sciences. Representatives from the energy and 
environmental industries as well as academia and 
government gathered at the Indiana University 
Geological Field Station (IUGFS) in a community¬ 
wide meeting on the future of field-based 
education in August of 2006, universally (but 
anecdotally) acknowledged the central 
importance of undergraduate field education, 
especially in capstone field camp settings. Even 
when graduates of these programs went into lines 
of work that did not require field skills per se, 
those present at the meeting noted that problem 
solving skills and habits of mind such as actively 
using multiple working hypotheses were greatly 
increased from geologic field camp education. 

The geological sciences are among the most 
visually oriented of all sciences, and the earth 
sciences as a collection of allied fields collectively 
relies on spatially integrated and spatially 
embedded data more than any other branch of the 
natural sciences (Chadwick, 1978, Kali and Orion, 
1996). The field clearly makes these spatial 
relations more concrete, but also demands 
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abstract spatial thinking as well. Only recently 
have science education researchers begun to focus 
on the unique aspects of problem solving and 
cognition, and geologically specific spatial 
abilities that characterize expertise in this area, 
particularly in the critical outdoor teaching, 
learning and research environment (e.g. Orion, 
2003; Ishikawa & Kastens, 2005). 

The outdoor field-based environment is not 
ideally suited for controlled tests of cognition as 
would normally be carried out in laboratory or 
classroom educational setting, as the variables 
involved are many and human factors (i.e. human 
response to terrain, exhaustion, discomfort, etc.) 
become involved. Therefore the study of problem 
solving skills needs to be treated by proxy 
measures, and needs to explicitly work with the 
study of problem solving and decision making as 
it happens in natural, real-world settings. We 
have developed a methodology for analyzing 
navigational choices recorded on GPS units worn 
by students during field examinations that we 
have demonstrated reflects problem solving 
stages as defined by some workers in the 
cognitive science research fields of Naturalistic 
Decision Making (Klein, Orasanu, Calderwood, & 
Zsambok 1993; Marshall, 1995; Endsley, 2001). 
We discuss this approach and our methodologies 
in the sections that follow. 
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BACKGROUND 

Field-Based Learning in Deformed 
Sedimentary Sequences 

Most field camp courses have common 
curricular components that have students 
working with deformed sedimentary units. While 
field studies in metamorphic or igneous terranes 
are equally valuable to the well-rounded 
education of any undergraduate geologist, 
optimal controls for educational research of the 
type presented here are difficult to establish. 
Fortunately, a large portion of field education is 
based on structural and sedimentological 
problems in sedimentary rocks, and many of the 
field areas used for the instruction of 
undergraduate geology majors share a number of 
features which provide constraints which 
facilitate the study of student learning. These 
kinds of problems tend to have a highly 
deterministic geometry which allows prediction 
of subsurface structure from surface information, 
and prediction of the likely surface exposure in as- 
of-yet unmapped areas. This type of geologic 
problem lends itself well to testing by multiple 
geometric working hypotheses that can be tested 
by planned traverses of a field area optimized to 
search for data that confirms or rejects 
hypotheses. The proposition that underlies our 
research approach is that the navigation decisions 
made by students while investigating this type of 
field problem reflect their internal problem 
solving approaches as they fit testing and 
verification strategies derived from their mental 
models to traverse plans. 

Problem Solving 

A central issue emerges in the framing of this 
research, namely what is problem solving? And 
what is geologic problem solving relative to other 
kinds of problem solving in science? In many 
sciences and in mathematics, especially at the 
undergraduate level, students are confronted with 
analytical problems, where they are basically 
given puzzles to solve. In front of them lies all of 
the necessary data to completely solve the 
problem, and the primary challenge to the learner 
is to draw on their recognition of common 
patterns to put these pieces together to find the 
solution. Pattern recognition is a hallmark of 
"expert behavior" as described in Bransford, 
Brown, & Cocking (2000), but the primary 
distinction here is that all data is available at the 
outset. 

Geologic problem solving occasionally does 
involve this style of investigation, but in the field 
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this is rare. Field problem solving involves a full 
range of navigational skills (Schofield & Kirby, 
1994; Kozhevnikov & Hegarty, 2001) and map 
reading and many discrete skills related to spatial 
visualization (Ishikawa & Kastens, 2005). Beyond 
having to bring many separate skills to bear on 
geologic mapping situations, and assuming even 
complete mastery of these requisite skills, 
problem solving in the field also involves 
operating in fundamentally data-poor and under¬ 
determined situations. Researchers (and students 
who function as researchers during exercises and 
field exams) never have all the information they 
need to fully solve any given problem with the 
confidence of an analytical solution. They must 
rely on the construction of multiple working 
hypotheses which can be pursued in order to 
gather more relevant data which in turn improves 
the working hypotheses. This iterative, boot¬ 
strapping situation does also depend on expertise, 
but involves additional skills of planning and 
field navigation designed to optimize a path 
through likely data-rich regions in a field area 
given ever-present time constraints. Because of 
these additional features to problem solving in a 
geologic field context, it is clear that the act of 
problem solving is best studied in its naturalistic 
context and is impossible to study and duplicate 
in a fully controlled laboratory because so much 
of the problem solving strategy is bound up in 
individual response to the real situation. This is 
what leads us to the research traditions of 
Naturalistic Decision Making, and the conclusions 
that lead from this approach that allow us to 
understand how individuals of varying expertise 
tend to behave and are best taught to respond to 
variable field conditions. 

Naturalistic Decision Making 

Our theoretical approach to this study is 
rooted in the body of research in the cognitive 
science field of Naturalistic Decision Making 
(NDM). This area of research deals with problem 
solving and decision making in data-poor 
situations, usually under time constraints, where 
the presence of expertise has a strong influence on 
moment-to-moment decisions made by problem 
solvers. NDM researchers typically study the 
real-time decision making pathways and 
processes of professionals such as firefighters, 
military commanders, air traffic controllers, 
manufacturing production managers and any 
other class of decision makers who have to make 
relatively rapid decisions with incomplete 
information. In data-poor situations, success 
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relies on matching multiple emergent patterns to 
data as it comes in. Expert problem solvers 
employ pattern recognition to make an educated 
guess at the "class" or "style" of a situation and 
make decisions for gathering additional data that 
quickly reduce the number of possible solutions 
and constrain the true nature of the problem. In a 
geologic setting, expertise is required to visualize 
and interpret these features from exposed 
features, and the evolution of novices into experts 
in this field is poorly understood. Studies of 
teaching and learning in the field, must also 
consider the complex interactions of factors that 
may have a bearing on an individual student's 
actions, decisions, mental model formation, and 
ultimate learning outcomes. 

NDM is an appropriate context for geoscience 
education research especially in a field setting 
because it places an emphasis not only on the 
cognitive tasks and actions of an individual or 
group, but also on the context in which that 
person or group of people is responding to the 
task at hand - in this case the geology, exposure 
and topography of the field area itself. 
Furthermore, this area of decision-making 
research is all the more appropriate to the study of 
field-based geologic problem solving because of 
its emphasis on systematic observations and 
analysis of skilled decision makers in naturalistic, 
i.e. not experimental, conditions. Experimental 
settings are often not able to adequately 
reproduce all aspects of a field situation that affect 
real-time cognition and decision making. This 
general area of research is more suited to decision 
making processes informed by intuition rooted in 
expertise rather than analytical, algorithmic, or 
optimization types of solutions (Lipshitz, Klein, 
Orasanu, & Salas, 2001). This captures much of the 
difference between geological investigations, 
exercises and examinations in the field at the 
advanced undergraduate level as compared with 
physics or chemistry problem solving at this level. 

The recent work published by Claire Bond 
and colleagues in GSA Today (Bond, Gibbs, 
Shipton, & Jones, 2007) underscores the central 
role of expertise in interpretation of geologic data. 
Their work shows that experience can strongly 
bias interpretations of incomplete data, in their 
case a synthetic seismic data profile generated 
from a 2D forward model of deformation of 
sedimentary strata. The researchers knew the 
"right" answer - a situation almost never 
encountered under real field or exploration 
conditions - and had over 400 volunteer 
geoscientists with some level of experience in 


seismic interpretation produce their best 
interpretation of the work. Not surprisingly, 
experts carried their past experience into their 
problem-solving processes, but interestingly, 
those experts who drew on many experiences and 
techniques useful in multiple contexts tended to 
produce better answers. While this study does 
not bear directly on our results, it is absolute 
confirmation that the assumptions and 
approaches of Naturalistic Decision Making are 
valid and useful in understanding the work of 
professional geoscientists in field or exploration 
settings, and in developing best practices in 
undergraduate education that lead to that 
expertise. 

Many process models have been proposed 
within the NDM tradition (Klein et al., 1993; 
Zsambok & Klein, 1997). Out of these 
characterizations of problem solving and decision 
making in naturalistic settings, we find that the 
schema model of Marshall (1995) to be most 
productively adapted to geologic problem 
solving. This model recognizes that decision 
making involves the construction of mental 
models (schema) that are in turn constructed of 
subordinate schema that work together iteratively 
to provide the basis for decision making. 

Marshall's research group worked primarily 
with U.S. Navy battlefield tacticians whose 
interface with the units on the ground was a large 
computer display station and active command 
and control communications. Subjects were 
monitored by eye-tracking equipment, which 
recorded the path that tacticians' eyes followed 
across the displays during various scenarios. 
Coupled with this was analysis of 
communications, subsequent interviews, and in 
more recent work, pupil dilation vibration rate, 
which the group showed to be related to 
concentration intensity (Marshall, 2002). From 
analysis of eye tracking patterns correlated with 
concentration intensity and other communications 
evidence of effective (or not effective) problem 
solving, Marshall's group produced a model with 
four components that iteratively work together to 
construct problem solving in these types of data- 
poor, time-limited situations that they called a 
schema model, based on the schema or mental sub¬ 
models that had to go into the overall problem¬ 
solving process. 

Marshall's conceptualization of schema in 
problem solving includes the following four 
elements: 

Identification knowledge - the ability to 
recognize relevant information and assess from 
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clues in the environment when a situation is 
similar to prior experiences or education 

Elaboration knowledge - the immediate 
associated recall of related facts and elements 
which aid in the confirmation or adjustment of 
the initial assessment of a situation from 
identification knowledge. This is similar to the 
“ chunking" of information common to experts 
reviewed in Bransford et al. (2000). 

Planning knowledge - the ability to draw 
inferences and estimates, create goals and plans 
using the framework provided by identification 
and elaboration knowledge. 

Execution knowledge - the ability to utilize 
skills and procedures as needed to provide 
further information or take additional action to 
further a solution. 

These are iterative, interactive portions of the 
larger mental model (schema) that a decision 
maker uses to recognize emerging situations and 
direct current actions and future data collection 
priorities, but the absence of any of them prevents 
effective problem solving. This is easily applied 
to geologic problem solving, especially in field 
mapping and structural problem solving under 
time constraints. In the field, geologists identify 
rocks and make relevant measurements, elaborate 
through multiple working hypotheses explaining 
how these data are fit by larger-scale solutions, 
make plans to traverse the landscape to most 
efficiently test these hypotheses, and then execute 
the plans safely as terrain conditions allow. 
Clearly these individual steps are repeated as 
needed at many temporal and spatial scales as 
new data is revealed during a field traverse. 

From an educational research perspective, 
typically only the identification and execution 
steps are easily externalized. From checking 
completed geologic maps, notes and other direct 
observations in the field, it is clear if an 
identification is correct, and by tracking 
navigation, the execution step is recorded. Close 
analysis of patterns in the navigation data, along 
with field notes and finished geologic maps can 
be used to infer the quality of elaboration and 
planning knowledge in a field teaching setting, 
but we will show in this study that navigational 
patterns can also be analyzed to shed light on 
these otherwise internal, mental processes. 

FIELD OBSERVATIONS AND DATA 
COLLECTION 

This study was conducted in an Advanced 
Field Geology course for undergraduate geology 



Figure 1. Generalized geologic map of test area. 

The field area is approximately 1 km in width. 

Bounded area is approximately 2 km 2 , contour 

interval is 40 ft. 

majors, conducted in the contractional belt of the 
northwestern San Gabriel Mountains near Frazier 
Park, CA. Students in this course had completed 
beginning and intermediate semester-length field 
courses and were all long-time residents of 
Southern California and were accustomed to steep 
topography, typical field conditions in the region 
and the general geologic and tectonic history of 
the region. The first author was the also the 
instructor for this course, and had been the 
instructor for many of these students in their 
introductory field course, so care was taken to 
secure informed consent between the second 
author and student participants in compliance 
with our IRB approval. The first author had no 
knowledge of which students had elected to 
participate in the study in advance, and no data 
analysis was conducted by anyone on the research 
team until after the course was completed. We 
acknowledge that these authors did have insights 
into individual student histories and tendencies 
which likely influenced some of our 
interpretations, although as will be shown below, 
efforts were made to reduce this effect by 
triangulating objective and subjective measures in 
forming conclusions. 

Students worked in pairs or groups of 3 for an 
exercise in a given region for a week, and then 
completed an all-day independent field 
examination in a nearby location, in the same 
sequence of rock units and exhibiting a similar 
structural style. Students were instructed not to 
communicate or follow one another, and were 
monitored for compliance. Students had 
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Student 

Pseudonym 

Map Score 

Polygon 

Score 

General 

Pattern 

Primary Codes (Speed) 





Fast 

Measured 

Slow 

Static 

Adrianne 

33 

47 

Figure 8 

10 

5 

13 

5 

Jay 

24 

50 

Circular/ 

loop 

10 

12 

8 

2 

Julie 

24 

44 

Circular/ 

loop 

2 

12 

8 

11 

Bob 

18 

48 

Zigzag 

7 

7 

10 

8 

Bill 

15 

39 

"E" pattern 

7 

12 

10 

2 

Jesus 

13 

39 

Stochastic 

7 

6 

11 

10 

Mark 

12 

44 

Circular/ 

loop 

11 

9 

4 

5 

Jesse 

3 

40 

Pinched loop 

3 

7 

13 

8 









Student 

Pseudonym 

Map Score 

Polygon 

Score 

General 

Pattern 

Secondary Codes ( Sequence and Reoccupation) 





Double Back 

Retrace 

Touch & Go 

Path Cross 

Adrianne 

33 

47 

Figure 8 

0 

0 

0 

4 

Jay 

24 

50 

Circular/ 

loop 

3 

3 

0 

0 

Julie 

24 

44 

Circular/ 

loop 

0 

2 

0 

0 

Bob 

18 

48 

Zigzag 

5 

2 

2 

0 

Bill 

15 

39 

"E" pattern 

4 

3 

0 

0 

Jesus 

13 

39 

Stochastic 

1 

1 

2 

2 

Mark 

12 

44 

Circular/ 

loop 

3 

3 

1 

1 

Jesse 

3 

40 

Pinched loop 

4 

3 

2 

0 


Table 1. Combined quantitative data for all participants. The Map and Polygon scores are 
determined as discussed in the text. The number of primary and secondary codes are summarized 
from track analysis for each student. Code descriptions and illustrations are presented in Figure 4. 


approximately 7 hours to produce a geologic map 
from available field data in a bounded region 
roughly 2 km 2 in size, shown in Figure 1. The 
area included sufficient exposure of deformed 
sedimentary rocks to find geological data but also 
had widespread tree cover and very steep 
topography at the southern boundary. These 
features necessitated intelligent planning of 
traverses across this landscape to maximize data 
collection and interpretation. 

Students were fitted with GPS units set to 
record their location every 3 minutes for the entire 
test period. They were allowed to look at these 
units, but their base maps deliberately had no 


geo-referencing information, rendering the units 
useless for navigation. Students were also 
instructed to make stations at data collection 
locations and record raw data both on the base 
maps and in their field notes. By the end of the 
test period the students were expected to hand in 
a completed geologic map of the region along 
with notes containing their raw data and 
illustrating their ongoing thought processes 
throughout the exam. We were able to recover 
and analyze data from only eight of our 15 
participants in this field area due to loss of 
satellite coverage or power failure for some GPS 
units. 
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Figure 2. Generalized geology with hotspots. 


work for that student. Adjacent polygons were 
strung together in a sequence to create time-series 
tracks. This data-processing approach enabled 
two kinds of complementary analyses. Polygon 
scores were assigned based on the degree to 
which polygons intersected hotspots, and a 
second coding scale that captured the dynamic 
qualities of the tracks. The polygon scoring scale 
(Figure 3) is a rubric-style measure that runs from 
zero to three depending mostly on the degree and 
duration of intersection of a student's navigation 
track with known critical locations (hotspots). A 
score of 3 means that the polygon completely 
engulfed a hotspot, indicating that the student 
stopped and investigated this region. Scores 
decrease with evidence that a student made a 
glancing pass of a critical area or traversed 
through a critical area too fast to be confident they 


FIELD NAVIGATION DATA ANALYSIS 

We scored all of the student maps against a 
traditional rubric constructed for evaluating 
geologic maps in field instruction, referenced to a 
map constructed by course instructors for this 
examination area. Points were awarded for 
accurate recognition and placement on the map of 
key geologic features such as structural elements 
(e.g. fold axes, etc.), contacts between geologic 
units, and correct identification of formations. 
Decreasing amounts of points for each key feature 
were granted with decreasing accuracy of location 
or omission of that feature. The student map 
scores are reported in Table 1, along with results 
of subsequent navigation track coding. 

All complete GPS records were imported into 
ArcGIS for analysis. Density clustering produced 
"hotspots", or contoured plots showing frequently 
visited locations where students had stopped 
throughout their traverses. These are shown 
superposed on the geology as mapped by the 
authors in Figure 2. The field area was later 
rechecked by investigators, and we discovered 
that these hotspot locations did indeed correlate 
with key geologic features. It is clear that these 
often-visited locations were necessary stations for 
students to visit in order to gather critical data. 
This does not mean that students were able to 
understand the significance of what they had 
found at these locations, but it does show that 
students were drawn to the same significant 
features. This observation is explored further 
below in the general discussion of tracks and track 
analysis. 

We constructed polygons of each consecutive 
five GPS data points, representing 15 minutes of 


meaningfully investigated the region. A score of 0 
was assigned for the polygons that did not 
intersect hotspots at all. Students were only 
scored once for duplicate passes across locations. 
The polygon scores are also reported in Table 1. 

To capture and describe the temporal 
dynamics of the track sequences, we settled on a 
two-part coding scheme that lent direct insight 
into the dimensions of Marshall's schema model 
that undergird our study. The four primary codes 
are related to travel speed, and each polygon in 
the time sequence was coded as fast, measured, 
slow or static. The code choice was tempered by 
knowledge of the terrain, such that a small 
polygon on steep terrain could be labeled as fast if 
it was relatively larger than other nearby 
polygons. The six secondary codes were related 
to sequences of polygons and reoccupation of sites 
in the field area. This approach enabled us to 
code track sequences such as "double back" 
maneuvers or star-shaped sets of polygons with a 
common origin that suggested repeated 
investigatory forays from a single starting point in 
a region, called a "touch and go". Our full coding 
scheme is illustrated in Figure 4. We coded these 
in time-series fashion from start to finish of the 
field examination, and totaled the number of 
instances of each code for each student. These 
data are also reported in Table 1. 

Semi-structured, stimulated-recall style 
interviews were conducted with selected students, 
and they were asked to explain their mapping 
sequence, results, and thought processes using 
their finished map and notes. These interviews 
and analyses are reported in Lieder (2005) and 
Lieder and Riggs (2004). The students were shown 
their traverse patterns and the hotspots. Students 
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Figure 3. Polygon 
scoring rubric. 


were asked to talk about what they investigated 
throughout the exam, and were probed to 
elucidate their problem solving techniques. 
Interviews allowed us to independently calibrate 
our interpretations of track patterns and our track 
coding to the students' account of their problem 
solving, and allowed us to understand which 
aspects of the coding scheme indicated success or 
failure of the various stages of problem solving. 

FINDINGS 

Student Navigation Track Results 

We present below all the individual results 
from this investigation, including each complete 
polygon track, the corresponding coding time- 
series (each polygon is shown by the range of the 
5 GPS points used to assemble it, 1-5, 6-10, etc.), 
and their finished geologic maps. Unfortunately, 
the print medium does not permit the dynamic 
presentation of all students' polygon tracks 
accompanied by real-time coding illustration. To 
augment this static presentation of our data, we 
have posted animated versions of these figures on 
the Internet which show the temporal progression 
of each student's traverse. These animations are 
available by navigating to links for Research/ 
Field Navigation Studies available at the Riggs 
Group web pages at http://www.purdue.edu/ 
eas/riggslab. Below we have attempted to 
provide a relevant summary narrative for each 
student in the study, with analysis of relevant 
major points observed in each students' behavior 
in the field. All relevant raw data and coding data 
are summarized in Tables 1 and 2 and presented 
graphically in Figures 5-12. 

Students with Above Average Map Scores 

Adrianne (Map Score: 33 out of 38, Figure 5) 

Analysis of Adrianne's polygon track 
produced very few secondary codes and the 
primary codes indicated that Adrianne alternated 
between fast and slow paces. The "Path Cross" 
code appeared four times and each time was 
paired with a "fast" pace primary code indicating 




Figure 4. Dynamic codes for polygon navigation 
tracks - see full description which follows. 
Numbered polygons in the secondary codes 
indicate temporal sequence. Linear - Participant's 
movement is linear through the field area from point "A" to 
point "B". This can he broken down into 3 sub-codes based in 
the speed at which the participant moves: Fast; Normal or 
Slow linear. Designation of the sub-code is qualitative. Static- 
Participant shows little or no movement for a time span 
exceeding 15 minutes (1 polygon). Polygon is very small or 
non-existent. Double Back- Participant retraces previous 
polygon; consecutive polygons overlap to a high degree. Back 
and Forth - Similar to a "Double Back", but with an extra 
retrace, or several retraces; participant moves from "A" to 
"B", "B" to "A", and then back to "B", and so on. All retraces 
occur on consecutive polygons. Retrace - Similar to a "Double 
Back", but the timing is different. Participant retraces a 
previously occupied region, but not on consecutive polygons. 
Touch and Go - Participant "touches" a previously occupied 
area and on the consecutive polygon moves out (at an angle) to 
a new area. Branching - Participant moves to a point (A) 
moves ~ 90° linearly to "B", immediately returns to "A" and 
then continues on a straight line (from "B" through "A") to a 
new area/point "C". Path Cross - Participant intersects or 
bisects a previous path perpendicularly and continues across it 
into a new area. 

that she moves through these reoccupied areas at 
a fast pace, only doing so to cross into new un¬ 
occupied areas. 

Adrianne's overall traverse in the field 
included several looping maneuvers and this 
strategy led to the occurrence of these path 
crosses. It's also clear from the tracks that 
Adrianne spends a significant amount of time 
outside of the field exam area and this doesn't 
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Figure 5. Adrianne's navigation track, completed map, and coding sequence. 
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Figure 6. Jay's navigation track, completed map, and coding sequence. 
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Figure 9. Bill's navigation track, completed map, and coding sequence. 
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Figure 10. Jesus' navigation track, completed map, and coding sequence. 
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Figure 11. Mark's navigation track, completed map, and coding sequence. 
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Name 

Map Score 

2nd codes 

Adrianne 

33 

4 

Jay 

24 

8 

Julie 

24 

4 

Bob 

18 

9 

Bill 

15 

7 

Jesus 

13 

12 

Mark 

12 

12 

Jesse 

3 

9 

Average 

17.75 

8.125 


Table 2. All students with above average map 
scores have a lower than average total of 
secondary codes, except for Bob. 

seem to affect her performance as indicated by her 
map score as she received the highest map score. 
As shown by the lack of secondary codes and her 
completed map, Adrianne maximizes her time in 
the field area and a majority of her time was spent 
near significant geologic structures which she 
correctly identifies. 

Jay (Map Score: 24 out of 38, Figure 6) 

Jay had one of the most orderly traverses 
through the field as he completed one full 
clockwise circuit. Jay's pace was either measured 
or slow, until the end of his traverse when it 
became fast. There were very few secondary codes 
throughout most of the track until the very end, 
which coincides with the increased pace as noted. 
The secondary codes include "back and forth", 
"retrace", and a "double back", which all 
represent maneuvers in which he reoccupies areas 
that he had been in at the beginning of his 
traverse. All but one of these codes occurs after 
Jay has already made a full traverse of the entire 
area; the one code that occurs before this is a 
double back. Also at the beginning of Jay's 
traverse he uses approximately 45 minutes to 
presumably get to high topography. Jay was 
accountable for all the major structures in the 
field, which is likely related to the efficient path 
that Jay takes through the field, but he was 
marked off on his completed map because the 
placement of the structures were inaccurate. 

Julie (Map Score: 24 out of 38, Figure 7) 

Similar to Jay, Julie had a very orderly 
traverse that was one large clockwise loop 
through the field area. Julie's pace was very 
measured or slow and there were several 
occasions where she was static for -10-25 minutes 
at a time. Like Jay, Julie had very few secondary 
codes indicating a reoccupation of an area and 


these didn't appear till the very end of the 
traverse; one of the retrace and double back codes 
are proceeded by out of bounds maneuvers. Julie 
was also able to identify all the major structures in 
the area, but she struggled with the placement, 
trend, and sense of these structures accounting for 
the point deductions on her completed map. 

Bob (Map Score 18 out of 38, Figure 8) 

Bob initially starts out in a clockwise circular 
pattern, but his traverse quickly becomes 
inefficient as he executes a zigzag pattern up and 
down steep terrain throughout the field area. 
Overall Bob's pace was evenly distributed 
between all 4 primary codes, although in the 
beginning his pace was mostly slow to moderate, 
while at the end of his traverse it was fast or static. 
Bob had several secondary codes which start 
approximately half way through his traverse and 
after he has started the zigzag portion of his 
traverse. The touch and go codes and double back 
codes are a result of Bob's zigzag pattern as he 
returns to the valley before heading back up the 
mountain and back down. The retrace codes 
associated with Bob's traverse are at the very end 
of his traverse as he returns to the beginning 
where he remains static, before returning to the 
initial starting point. Like his higher scoring peers. 
Bob identifies all the important structures, but his 
lack of precession in placing the structures and 
identifying the correct sense of displacement leads 
to a reduction in points. 

Students with Below Average Map Scores 

Bill (Map score: 15 out of 38, Figure 9) 

Bill's traverse is the only non-circular/loop 
pattern that appeared to be planned as such. Bill's 
traverse looks like a 3 pronged approach as he 
starts out by heading west, heads uphill, comes 
back down to the valley and heads to the far east, 
where he heads up terrain again before back 
coming down, heads down the valley, stops 
halfway between the previous 2 approaches and 
heads uphill, before finally returning to his 
starting point. Bill is constantly moving as he is 
only static at the beginning and the very end and 
his pace is evenly distributed between slow, 
moderate, and fast. The majority of Bill's 
secondary codes (Retraces and several Double 
backs) are the result of Bill's planned navigation 
pattern. Bill is the first student to not locate all the 
important geologic features and this is reflected in 
his map score; his lower score is also the result of 
his failure to accurately locate the features he did 
identify on the map. His failure to locate all the 
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structures is likely a result of his navigation 
pattern, because he did not visit areas that the 
others did who had more complete patterns. 

Jesus (Map score: 13 out of 38, Figure 10) 

Jesus had one of the most complex navigation 
patterns (right) and his maneuvers appear 
relatively random and unplanned. Jesus' pace was 
generally very slow in the beginning and end, 
while his pace was rapid in the middle of his 
traverse. Jesus had the highest amount of 
secondary codes (tied with Mark) spread out 
evenly throughout the navigation; the high 
number of secondary codes are likely related to 
the complex, random, and unplanned nature of 
Jesus' traverse. Jesus also had the highest variety 
of codes (double back, retrace, branch, touch and 
go, and path cross), which is also related to the 
complexity of his navigation pattern. 
Significantly, he also has a one-hour static period 
near the end of the examination time spent near 
the isolated data point on his map in the northeast 
sector of the field area, suggesting a total state of 
confusion and/or a subject that has completely 
given up on the exam. This is also seen in the 
work of Marshall (2002, and personal 
communication) in eye tracking paths of 
battlefield tacticians on tactical displays who have 
effectively surrendered or become mired in 
confusion. Jesus' below average score is a result of 
his failure to identify the fold in the valley or the 
tear faults in the southern portion of the field 
exam area. Looking at Jesus' tracks it's evident 
that he did not visit the areas where these 
structures were. 

Mark (Map score: 12 out of 38, Figure 11) 

Mark's traverse looks similar to circular/loop 
patterns of the higher scoring students, but is 
smaller and is not efficient. Mark's primary codes 
indicate that he moved at a rapid pace (the fastest 
of anyone). Most of his secondary codes occurred 
towards the end of his traverse and look to be 
related problem solving issues as he continually 
reoccupies the valley for the last 2+ hrs of his 
traverse. This rapid back-and-forth movement 
over previously investigated ground has also been 
seen commonly in Marshall's eye tracking work 
(2002 and personal communication), and is 
interpreted as a sign of confusion. Subjects in her 
studies will commonly glance back and forth 
repeatedly over known data they do not 
understand, hoping to literally see something 
new. Mark has demonstrated precisely the same 
behavior, but by walking repeatedly over the 


same ground on a much larger physical scale than 
any eye tracking exercise. Mark has a couple of 
secondary codes at the beginning of his traverse, 
but these are more likely related to terrain issues 
than problem solving ones. Mark's low score on 
the exam is because he didn't identify the tear 
faults and the location of the thrust fault is poorly 
placed, this is likely related to his lack of time 
spent where those structures are located. 

Jesse (Map score: 3 out of 38, Figure 12) 

Jesse had a circular/loop pattern through part 
of the field area. Jesse's primary codes indicate 
that he moved very slowly through the field area 
and in fact had the slowest pace. Most of Jesse's 
secondary codes occur at the beginning and the 
end when he's not moving in a looping traverse. 
Even though his traverse is similar to some of the 
higher scoring students, it is less efficient as he 
doubles back and retraces at the beginning and 
ends causing the loop to "pinch off", which causes 
Jesse to miss large sections of the field exam area. 
Jesse missed several key structures, including the 
east tear fault, the fold, and the majority of the 
thrust, leading to a severe reduction in his map 
score. 

DISCUSSION 

We found only a loose correlation of map and 
polygon scores in our data, and there were many 
instances where students with the same polygon 
score had widely varying map scores, suggesting 
that this measure alone did not explain the range 
of performance observed. However, the two-part 
track coding scheme presented allowed a finer- 
grained analysis of the navigation tracks and 
yielded many insights into problem solving steps 
and helped more completely link navigation 
characteristics to performance. 

As can be seen in Tables 1 and 2, a lack of 
secondary codes is a substantial predictor of high 
performance, especially when the primary code is 
taken into consideration. For example, the fact 
that Adrianne's path cross codes were paired with 
a fast-paced primary code (see Figure 5), shows 
that even though she did spend time in previously 
visited areas she spent very little time there. 
Adrianne's path crosses indicate that she may not 
have initially had a navigational plan, but the low 
amount of time spent reoccupying areas indicate 
that she was successfully able to identify and 
elaborate on information in those initial areas and 
permitted her to quickly form a plan, thus 
carrying out the planning and execution steps. 
The successful implementation of the full schema 
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framework is reflected in Adrianne's map score. 
In contrast, the nature of Jesus' path crosses (see 
Figure 10) indicates a deeper problem solving 
issue. The difference is inferred from the student's 
overall pattern: Adrianne's organized traverse 
versus' Jesus' random traverse. 

Further insight to the application of 
Marshall's schema model is given by the generally 
high polygon scores (Table 1) and the coincidence 
of hotspots with significant geologic features 
shown in Figure 2. Students were monitored by 
the authors during test conditions to make sure 
they were not following one another or 
collaborating, and the terrain and vegetation 
cover in this area is sufficiently complex and 
dense to prevent casual observation of one 
student's movements by another. We are 
reasonably confident that students reached these 
locations on their own guidance and stopped at 
these locations because of their own recognition of 
geologically significant information at these 
locations. This represents successful completion 
of the identification step in naturalistic decision 
making by virtually all of the students, which 
would be expected of advanced geology 
undergraduates with the amount of prior field 
experience these students had. The large variance 
in map scores, coupled with insights from our 
track sequence coding, indicates that the 
subsequent failure or success of problem solving 
follows largely from a failure to elaborate 
successfully on the information gained, and/or a 
misinterpretation of the information gained at 
these sites that led to erroneous elaboration into 
mental models of the field area that could not be 
substantiated with further investigation. Student 
interviews and analysis of notes supports this 
interpretation, as those students who had poor 
final maps also tended to have little evidence of 
planning and model elaboration in their recorded 
field notes. 

A distinction should be made between those 
students without clear, emergent navigation 
plans, the ones with efficient plans, and those 
with inefficient plans. The appearance of a plan 
does not necessarily correlate to above average 
scores; Jay, Julie and possibly Adrianne, who all 
evidently developed navigation plans scored 
above average, but in contrast students like Bill, 
Bob, Jesse, and Mark who also had plans, 
performed below average because their plans 
were also largely inefficient. Certain types of 
traverses will cover just as much area as others, 
but will be physically exhausting which could 
possibly tax mental facilities, leading to a 
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decreased map score. The formulation of an 
inefficient plan (Bill, Bob, Jesse, and Mark) or no 
plan at all (Jesus) represents a breakdown in the 
stages of problem solving. Analysis of the 
individual secondary codes reveals that 
significant amounts of time (in some cases hours) 
were spent by these 5 students reoccupying areas. 
Additionally, the occurrence and repetition of 
certain codes (back and forth and retrace) 
indicates a decrease in problem solving. For 
instance, Mark executes several back and forth 
maneuvers in the same area at the end of his 
traverse, suggesting that he was struggling with 
the geology in that area. There is a loose 
correlation between the number of secondary 
codes a student has and performance. The top 4 
map scorers had the lowest 4 secondary codes 
except for one instance (Jay), and the bottom 4 
map scorers had the highest 4 secondary codes 
except for one instance (Bill). 

General Conclusions from Track Analysis 

• Secondary codes are a result of a student's 
plan (or lack thereof) in the field, which could 
indicate how much active problem solving 
they are engaged in. In some cases, like 
Adrianne, path crosses do not seem to 
indicate a lack of problem solving ability, but 
just indicate a lack of an initial or consistent 
plan. In contrast, the nature of Jesus' path 
crosses indicates deeper difficulties with 
problem solving. The difference is inferred 
from the student's overall pattern: Adrianne's 
organized traverse versus Jesus' very random 
traverse. 

• Certain types of traverses will cover just as 
much area as others, but will be physically 
exhausting which could possibly tax mental 
facilities such as problem solving leading to a 
decreased map score. For example. Bob visits 
the same areas as his higher scoring peers, but 
he receives a lower score on his map. 

• The occurrence of certain codes (back and 
forth), or the repetition of a code, may 
indicate a decrease in problem solving ability. 
For instance, Mark executes several back and 
forth maneuvers in the same area at the end of 
his traverse, suggesting that he was struggling 
with the geology in that area; this is 
supported by the lack of structures and 
inaccuracy of his map in this area. According 
to Marshall (2002 and personal 
communication), this type of rapid back-and- 
forth movement in her eye tracking studies is 
also indicative of confusion, which establishes 
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an interesting parallel between her studies of 
behavior and cognition and this work. 
Discussions with the Marshall research group 
about this work (she co-advised Lieder, 2005) 
have also revealed striking similarities with 
patterns developed in eye tracking research 
that we have not discussed here. Our 
methods and approaches with track coding 
and polygon scoring have already flowed 
back from this study in to that research area, 
and the unexpected similarity of eye tracking 
and field navigation patterns with similar 
problem solving task characteristics but vastly 
different temporal and spatial scales offers a 
new comparative window into basic problem 
solving cognition and the role of expertise. 

Limitations and Questions 

This study contributes new, independent tools 
for measuring and monitoring field geology 
performance that do correlate with traditional 
map grading and which give higher resolution 
understanding of specific difficulties to geoscience 
education researchers working with field based 
learning at this advanced undergraduate level. 

However, there are a number of questions 
raised by this data set that we are continuing to 
investigate and which call for further 
investigations of this nature to be designed and 
conducted. A range of methodological, technical 
and research questions for future investigation are 
raised by this study, including: 

• How are navigation patterns effected by 
classroom educational background (prior 
preparation), prior field experience, personal 
background (outdoor comfort/experience), 
and social influences (used to group/ 
independent work)? 

• What does expert behavior look like using 
this same methodology? We are actively 
analyzing new data with beginning students, 
but it would be useful to know what the 
expert "goal state" looks like to calibrate our 
measurements. 

• Are issues of Novelty Space (Orion, 1993; 
Orion and Hofstein, 1994), the "newness" of a 
field camp setting relevant to performance in 
field examinations? 

• Human factors (athletic performance and 
related physiological issues) are likely 
relevant to this analysis, but we have no 
measure of these factors at this time other 
than our personal knowledge of the students 
involved in this study. 

• While we now have a good framework for 


forming research questions and analytical 
inquiries using this of this kind of data, we do 
not yet have a rich enough data set to test our 
methods for sensitivity in sampling rate, 
polygon methodology, and trustworthiness in 
primary and secondary maneuver codes. 
Deeper qualitative interviews and 
observations are needed, as are more 
advanced quantitative methods and larger 
data sets with a broader range of student 
participants. 

• We have no way to independently or 
quantitatively gauge how topography in the 
test area influenced navigation or traverse 
speed - additional well-known map test areas 
are needed to help constrain topographic 
influences, as are more quantitative 
approaches to terrain corrections for polygon 
size and speed analysis. 

• More rigorous methods for assigning polygon 
scores based on quantitative overlap are 
required. The quantitative spatial analysis 
methods and approach of Baker and Libarkin 
(2007), which build on the methods reported 
here, add significant new formalism to the 
hotspot generation process using geospatial 
analysis techniques. Collaboration between 
many research groups will be required to 
flesh out this approach and expand sample 
sizes to ensure robust conclusions. 

• How can one use this methodology in real 
time or near-real time to enhance student 
learning in field camp settings? Application of 
these research tools as assessment measures 
could significantly advance field education in 
the geosciences. 

CONCLUSIONS AND APPLICATIONS 

This study contributes new, independent tools 
for measuring and monitoring field geology 
performance that do correlate with traditional 
map grading and which give higher resolution 
understanding of specific difficulties to geoscience 
education researchers working with field based 
learning at this advanced undergraduate level. 

We have also shown in this study that 
application of insights and guiding frameworks 
from cognitive science research can be readily 
applied to geoscience field problem solving. Our 
methods show how navigation is related to 
problem solving, and additionally we have made 
the contribution of new coding schemes that are 
readily applicable to other field education settings 
in the geosciences and potential elsewhere. These 
tools provide useful new methods for 
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understanding the development of this type of 
expertise, and provide a new avenue for work by 
other researchers exploring various stages and 
settings of this poorly understood process. 

Broader interest in this study is based in the 
potential practical applications of these techniques 
to study any area of science education research 
that considers navigation through space coupled 
with problem solving. Studies in museum 
science, classroom dynamics, and other field 
science applications could potentially benefit from 
a modification and re-tasking of our approach. 
This method could also be adapted to real-time 
monitoring of student problem solving and 
progress in educational settings. 

Discussions with the Marshall research group 
about this work have also revealed striking 
similarities with patterns developed in eye 
tracking research that we have not discussed here. 
Our methods and approaches with track coding 
and polygon scoring have already flowed back 
from this study in to that research area, and the 
unexpected similarity of eye tracking and field 
navigation patterns with similar problem solving 
task characteristics but vastly different temporal 
and spatial scales offers a new comparative 
window into basic problem solving cognition and 
the role of expertise. 
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